Method and system for storytelling on a computing device via user editing

ABSTRACT

Disclosed is a method and system for enabling storytelling on a computing device. A processor analyzes a set of media items associated with the user, each media item having associated metadata. The processor identifies one or more related characteristics among the media items in the set to form a cluster of media items associated with an event associated with the user. The processor selects templates from a template database, each template configured to represent a moment in the event. The processor edits selected media items in the cluster to fit into the selected templates. The processor creates a mixed-media module comprising the templates organized into a desired sequence. The processor displays, to the user, a preview of the mixed-media module, receives edit commands for editing the module, and applies the edit commands to the module to create a final module. The processor then displays the final mixed-media module.

CROSS REFERENCE TO RELATED APPLICATIONS

This patent application claims the benefit of Provisional PatentApplication Ser. No. 61/739,669, titled “Systems and Methods InvolvingInformation Modules Including Auto-Publishing, Mobile Device and/orOther Features” filed on Dec. 19, 2012; Provisional Patent ApplicationSer. No. 61/739,698, titled “Systems and Methods Involving InformationModules Including Metadata/Meta signal/Shared Media Processing,Auto-Publishing, Mobile Device, Full Stack and/or Other Features” filedon Dec. 19, 2012; Provisional Patent Application Ser. No. 61/798,000,titled “Systems and Methods Involving Information Modules IncludingMetadata/Meta signal Processing, Auto-Publishing, Mobile Device and/orOther Features” filed on Dec. 19, 2012; Provisional Patent ApplicationSer. No. 61/740,347, titled “Systems and Methods Involving InformationModules Including Metadata/Meta signal Processing, Auto-Publishing,Mobile Device and/or Other Features” filed on Dec. 20, 2012; ProvisionalPatent Application Ser. No. 61/740,280, titled “Systems and MethodsInvolving Information Modules Including Metadata/Meta signal Processing,Auto-Publishing, Mobile Device and/or Other Features” filed on Dec. 20,2012; Provisional Patent Application Ser. No. 61/740,416, titled“Systems and Methods Involving Information Modules IncludingMetadata/Meta signal/Shared Media Processing, Auto-Publishing, MobileDevice, Full Stack and/or Other Features” filed on Dec. 20, 2012;Provisional Patent Application Ser. No. 61/761,675, titled “Systems andMethods of Capturing Metadata and Meta-Signals of Recorded Media onMobile Devices and Auto-Publishing Event-Based Media Into aSynchronized, Editable Multimedia Story with One Tap Storytelling” filedon Feb. 6, 2013, Provisional Patent Application Ser. No. 61/815,724,titled “Systems and Methods Involving Information Modules IncludingLayout, Capture, Camera, Display and/or Other Features” filed on Apr.24, 2013, and Provisional Patent Application Ser. No. 61/821,216, titled“Systems and Methods Involving Information Modules Including SmartScrolling, Layout, Capture, Camera, Display and/or Other Features” filedon May 8, 2013, all of which are incorporated herein by reference.

FIELD

The present disclosure relates to enabling storytelling on a mobilecomputing device, and more specifically to creating and displaying amixed-media module representing a story on a mobile computing device.

BACKGROUND

Current methods for cataloging and viewing multimedia stored on a mobiledevice is limited. The arrangement of the media items into any kind oflogical sequence or presentation requires manual intervention andanalysis of the individual files. Present solutions involve a variety ofdrawbacks, such as being too time intensive, and/or requiring excessiveor complex editing, among others. Further, as the user manually arrangesdifferent media items into a sequence, the user often loses sight of thehigh level theme that the user initially wanted to portray in thesequence.

SUMMARY

A method and system for enabling storytelling on a computing device aredisclosed. In one aspect, a method, system, and computer readablestorage medium for storytelling on a computing device operated by a userinclude the following steps performed by a processor of the computingdevice. The processor analyzes a set of media items associated with theuser, each media item having associated metadata. The processoridentifies, based on analysis of the associated metadata, one or morerelated characteristics among the media items in the set to form acluster of media items associated with an event associated with theuser. The processor selects, based on analysis of the media items in thecluster, a plurality of templates from a template database, eachtemplate of the plurality of templates configured to represent a momentin the event. The processor edits selected media items in the cluster tofit into the selected templates. The processor creates a mixed-mediamodule comprising the plurality of templates organized into a desiredsequence for the selected templates. The processor displays, to theuser, a preview of the mixed-media module, receives edit commands fromthe user for editing the mixed-media module, and applies the editcommands to the mixed-media module to create a final mixed-media module.The processor then displays the final mixed-media module.

In one embodiment, the processor analyzes the associated metadata ofeach media item in the set The processor can obtain the media items inthe set. The processor can analyze the media items in the cluster. Inone embodiment, the processor determines the desired sequence for theselected templates. The processor can display, to the user, editfunctions for editing the mixed-media module. The receiving of the editcommands can include, for example, receiving a pace of the mixed-mediamodule, receiving a song or music, and/or receiving one or more of acaption, a comment, and a title of the mixed-media module.

In one aspect, a method for and a server computer includes logic forcommunicating a software application to a computing device operated by auser, where the software application is configured to perform thesesteps.

In another aspect, a method, system, and computer readable storagemedium for storytelling on a computing device operated by a userincludes the following steps performed by a processor of the computingdevice. The processor analyzes a set of media items associated with theuser, where each media item has associated metadata. The processoridentifies, based on analysis of the associated metadata, one or morerelated characteristics among the media items in the set to form acluster of media items associated with an event. The processor selects,based on analysis of the media items in the cluster, a plurality oftemplates from a template database, where each template is configured torepresent a moment in the event. The processor edits selected mediaitems in the cluster to fit into the selected templates. The processorcreates a mixed-media module including the plurality of templatesorganized into a desired sequence for the selected templates.

In another aspect, a method, system, and computer readable storagemedium for storytelling on a computing device operated by a user includethe following: (a) analyzing, by a processor of the computing device, afirst set of media items associated with the user, where each media itemhas associated metadata; (b) identifying, by the processor, based onanalysis of the associated metadata in the first set, one or morerelated characteristics among the media items in the first set; (c)forming, by the processor, a cluster of media items associated with anevent based on the identified one or more related characteristics; (d)repeating, by the processor, steps (a) and (b) for a second set of mediaitems associated with the user; (e) adding, by the processor, one ormore media items in the second set having the one or more relatedcharacteristics, from a computing device associated with the second set,to the cluster of media items; (f) editing, by the processor, selectedmedia items in the cluster to fit into selected templates, each templateconfigured to represent a moment in the event; and (g) creating, by theprocessor, a mixed-media module comprising the plurality of templatesorganized into a desired sequence for the selected templates.

In another aspect, a method, system, and computer readable storagemedium for storytelling on a computing device operated by a user includethe following: (a) analyzing, by a processor of the computing device, afirst set of media items associated with the user, each media itemhaving associated metadata; (b) identifying, by the processor, based onanalysis of the associated metadata in the first set, one or morerelated characteristics among the media items in the first set; (c)forming, by the processor, a cluster of media items associated with anevent based on the identified one or more related characteristics; (d)repeating, by the processor, steps (a) and (b) for a second set of mediaitems, each media item in the second set having associated metadata andrelated to a social connection of the user; (e) adding, by the processorand based on the identifying, one or more media items in the second sethaving the one or more related characteristics, from a social media siteassociated with the second set, to the cluster of media items; (f)editing, by the processor, selected media items in the cluster to fitinto selected templates, each template configured to represent a momentin the event; and (g) creating, by the processor, a mixed-media modulecomprising the plurality of templates organized into a desired sequencefor the selected templates.

In another aspect, a method, system, and computer readable storagemedium for storytelling on a computing device operated by a user includethe following steps performed by a processor of the computing device.The processor analyzes a set of media items associated with the user,each media item having associated metadata. The processor identifies,based on analysis of the associated metadata, one or more relatedcharacteristics among the media items in the set to form a cluster ofmedia items associated with an event associated with the user. Theprocessor selects, based on analysis of the media items in the cluster,a plurality of templates from a template database, each template of theplurality of templates configured to represent a moment in the event.The processor edits selected media items in the cluster to fit into theselected templates. The processor scores each selected template based onthe metadata associated with the selected media items in the selectedtemplate. The processor creates a mixed-media module of predeterminedduration comprising the plurality of templates organized into a desiredsequence for the selected templates.

These and other aspects and embodiments will be apparent to those ofordinary skill in the art by reference to the following detaileddescription and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawing figures, which are not to scale, and where like referencenumerals indicate like elements throughout the several views:

FIG. 1 is a block diagram of a client device communicating with a servercomputer over a network in accordance with an embodiment of the presentdisclosure;

FIG. 2A is a flowchart illustrating steps performed by the client devicewhen executing a mobile app to perform one tap storytelling inaccordance with an embodiment of the present disclosure;

FIG. 2B is a block diagram illustrating the flow of the operations ofthe mobile app to create a mixed-media module in accordance with anembodiment of the present disclosure;

FIG. 3A is a flowchart illustrating steps performed by the client devicewhen executing the mobile app to perform one tap storytelling byobtaining media items from multiple sources in accordance with anembodiment of the present disclosure;

FIG. 3B is a block diagram of the client device to perform one tapstorytelling by obtaining media items from multiple sources inaccordance with an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating steps performed by the client devicewhen executing the mobile app to enable user editing on the mixed-mediamodule in accordance with an embodiment of the present disclosure;

FIG. 5 is a block diagram of a video engine used to create themixed-media module in accordance with an embodiment of the presentdisclosure;

FIG. 6 is a flowchart illustrating steps performed by the video engineto create the mixed-media module in accordance with an embodiment of thepresent disclosure;

FIG. 7 depicts one example of a schematic diagram illustrating a clientdevice in accordance with an embodiment of the present disclosure;

FIG. 8 is a block diagram illustrating an internal architecture of acomputer in accordance with an embodiment of the present disclosure;

FIG. 9 is a chart diagram showing story rank in accordance with anembodiment of the present disclosure;

FIGS. 10A, 10B, and 10C are block diagrams showing layouts or templatesused to create the mixed-media module in accordance with an embodimentof the present disclosure;

FIG. 11 shows a play mixed-media module screenshot of the mobile app inaccordance with an embodiment of the present disclosure;

FIG. 12 shows a preview screenshot of the mobile app in accordance withan embodiment of the present disclosure;

FIG. 13 shows an edit screenshot of the mobile app in accordance with anembodiment of the present disclosure;

FIG. 14 shows a create screenshot of the mobile app in accordance withan embodiment of the present disclosure;

FIG. 15 shows an invite screenshot of the mobile app in accordance withan embodiment of the present disclosure;

FIG. 16 shows a user invitation screenshot of the mobile app inaccordance with an embodiment of the present disclosure;

FIG. 17 shows a create with all screenshot of the mobile app inaccordance with an embodiment of the present disclosure;

FIG. 18 shows an event creation screenshot of the mobile app inaccordance with an embodiment of the present disclosure;

FIG. 19 shows a collaborative event creation screenshot of the mobileapp in accordance with an embodiment of the present disclosure; and

FIG. 20 shows a publication notification screenshot of the mobile app inaccordance with an embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

Embodiments are now discussed in more detail referring to the drawingsthat accompany the present application. In the accompanying drawings,like and/or corresponding elements are referred to by like referencenumbers.

Various embodiments are disclosed herein; however, it is to beunderstood that the disclosed embodiments are merely illustrative of thedisclosure that can be embodied in various forms. In addition, each ofthe examples given in connection with the various embodiments isintended to be illustrative, and not restrictive. Further, the figuresare not necessarily to scale, some features may be exaggerated to showdetails of particular components (and any size, material and similardetails shown in the figures are intended to be illustrative and notrestrictive). Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to variouslyemploy the disclosed embodiments.

The present invention is described below with reference to blockdiagrams and operational illustrations of methods and devices to selectand present media related to a specific topic. It is understood thateach block of the block diagrams or operational illustrations, andcombinations of blocks in the block diagrams or operationalillustrations, can be implemented by means of analog or digital hardwareand computer program instructions. These computer program instructionscan be provided to a processor of a general purpose computer, specialpurpose computer. ASIC, or other programmable data processing apparatus,such that the instructions, which execute via the processor of thecomputer or other programmable data processing apparatus, implements thefunctions/acts specified in the block diagrams or operational block orblocks.

In some alternate implementations, the functions/acts noted in theblocks can occur out of the order noted in the operationalillustrations. For example, two blocks shown in succession can in factbe executed substantially concurrently or the blocks can sometimes beexecuted in the reverse order, depending upon the functionality/actsinvolved. Furthermore, the embodiments of methods presented anddescribed as flowcharts in this disclosure are provided by way ofexample in order to provide a more complete understanding of thetechnology. The disclosed methods are not limited to the operations andlogical flow presented herein. Alternative embodiments are contemplatedin which the order of the various operations is altered and in whichsub-operations described as being part of a larger operation areperformed independently.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, the phrase “in one embodiment” as used herein does notnecessarily refer to the same embodiment and the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment. It is intended, for example, that claimed subject matterinclude combinations of example embodiments in whole or in part.

In general, terminology may be understood at least in part from usage incontext. For example, terms, such as “and”, “or”, or “and/or,” as usedherein may include a variety of meanings that may depend at least inpart upon the context in which such terms are used. Typically, “or” ifused to associate a list, such as A, B, or C, is intended to mean A, B,and C, here used in the inclusive sense, as well as A, B, or C, hereused in the exclusive sense. In addition, the term “one or more” as usedherein, depending at least in part upon context, may be used to describeany feature, structure, or characteristic in a singular sense or may beused to describe combinations of features, structures or characteristicsin a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again,may be understood to convey a singular usage or to convey a pluralusage, depending at least in part upon context. In addition, the term“based on” may be understood as not necessarily intended to convey anexclusive set of factors and may, instead, allow for existence ofadditional factors not necessarily expressly described, again, dependingat least in part on context.

FIG. 1 is a schematic diagram illustrating an example system 100 of anetwork and devices implementing embodiments of the present disclosure.Other embodiments that may vary, for example, in terms of arrangement orin terms of type of components, are also intended to be included withinclaimed subject matter. FIG. 1 includes, for example, a client device105 (also referred to herein as mobile computing device 105 or mobiledevice 105) in communication with a content server 130 over a wirelessnetwork 115 connected to a local area network (LAN)/wide area network(WAN) 120, such as the Internet. Content server 130 is also referred tobelow as server computer 130 or server 130. In one embodiment, theclient device 105 is also in communication with an advertisement server140, which may provide one or more advertisements to the client device105. Although shown as a wireless network 115 and WAN/LAN 120, theclient device 105 can communicate with servers 130, 140 via any type ofnetwork.

A computing device may be capable of sending or receiving signals, suchas via a wired or wireless network, or may be capable of processing orstoring signals, such as in memory as physical memory states, and may,therefore, operate as a server. Thus, devices capable of operating as aserver may include, as examples, dedicated rack-mounted servers, desktopcomputers, laptop computers, set top boxes, integrated devices combiningvarious features, such as two or more features of the foregoing devices,or the like. Servers may vary widely in configuration or capabilities,but generally a server may include one or more central processing unitsand memory. A server may also include one or more mass storage devices,one or more power supplies, one or more wired or wireless networkinterfaces, one or more input/output interfaces, or one or moreoperating systems, such as Windows Server, Mac OS X, Unix, Linux,FreeBSD, or the like.

Examples of devices that may operate as a content server include desktopcomputers, multiprocessor systems, microprocessor-type or programmableconsumer electronics, etc. Content server 130 may provide a variety ofservices that include, but are not limited to, web services, third-partyservices, audio services, video services, email services, instantmessaging (IM) services, SMS services, MMS services, FTP services, voiceover IP (VOIP) services, calendaring services, photo services, socialmedia services, or the like. Examples of content may include text,images, audio, video, or the like, which may be processed in the form ofphysical signals, such as electrical signals, for example, or may bestored in memory, as physical states, for example. In one embodiment,the content server 130 hosts or is in communication with a database 160.

A network may couple devices so that communications may be exchanged,such as between a server and a client device or other types of devices,including between wireless devices coupled via a wireless network, forexample. A network may also include mass storage, such as networkattached storage (NAS), a storage area network (SAN), or other forms ofcomputer or machine readable media, for example. A network may includethe Internet, one or more local area networks (LANs), one or more widearea networks (WANs), wire-line type connections, wireless typeconnections, or any combination thereof. Likewise, sub-networks, such asmay employ differing architectures or may be compliant or compatiblewith differing protocols, may interoperate within a larger network.Various types of devices may, for example, be made available to providean interoperable capability for differing architectures or protocols. Asone illustrative example, a router may provide a link between otherwiseseparate and independent LANs.

A communication link or channel may include, for example, analogtelephone lines, such as a twisted wire pair, a coaxial cable, full orfractional digital lines including T1, T2, T3, or T4 type lines,Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines(DSLs), wireless links including satellite links, or other communicationlinks or channels, such as may be known to those skilled in the art.Furthermore, a computing device or other related electronic devices maybe remotely coupled to a network, such as via a telephone line or link,for example.

A wireless network may couple client devices with a network. A wirelessnetwork may employ stand-alone ad-hoc networks, mesh networks, WirelessLAN (WLAN) networks, cellular networks, or the like. A wireless networkmay further include a system of terminals, gateways, routers, or thelike coupled by wireless radio links, or the like, which may movefreely, randomly or organize themselves arbitrarily, such that networktopology may change, at times even rapidly. A wireless network mayfurther employ a plurality of network access technologies, includingLong Term Evolution (LTE), WLAN, Wireless Router (WR) mesh, or 2nd, 3rd,or 4th generation (2G, 3G, or 4G) cellular technology, or the like.Network access technologies may enable wide area coverage for devices,such as client devices with varying degrees of mobility, for example.

For example, a network may enable RF or wireless type communication viaone or more network access technologies, such as Global System forMobile communication (GSM), Universal Mobile Telecommunications System(UMTS), General Packet Radio Services (GPRS), Enhanced Data GSMEnvironment (EDGE), 3GPP Long Term Evolution (LTE), LTE Advanced,Wideband Code Division Multiple Access (WCDMA), Bluetooth, 802.11b/g/n,or the like. A wireless network may include virtually any type ofwireless communication mechanism by which signals may be communicatedbetween devices, such as a client device or a computing device, betweenor within a network, or the like.

In one embodiment and as described herein, the client device 105 is amobile computing device such as a smartphone. In another embodiment, theclient device 105 is a tablet, laptop, wearable computing device (e.g.,a watch, clothing, sensor, band, glasses (e.g., Google® Glass), or anyother computing device. Although described herein with an embodiment ofa mobile computing device, it should be noted that the presentdisclosure is not limited to mobile devices or applications.

The server computer 130 (or other network-available resource) transmitsa mobile software application (or “app”) 150 to the client device 105.In one embodiment, the client device 105 downloads the mobile app 150(e.g., from an app store or from a web page). In one embodiment, theserver computer 130 transmits the mobile app 150 to the client device105. In one embodiment, the client device 105 includes the mobile app150 as part of (e.g., pre-installed) software resident on the clientdevice 105.

The client device 105 then executes the mobile app 150. Also referringto FIGS. 2A and 2B, in one embodiment the mobile app 150 analyzes a setof media items 250 associated with the user, where each media item hasassociated metadata (Step 205). Examples of the metadata can includecontext, time stamp or time frame, location (e.g., geo-location), tags,music or song(s) played before, during, or after the media item, and/orany other information associated with the media item. The processoranalyzes the associated metadata (shown with arrow 255 in FIG. 2B) ofeach media item in the set 250 (Step 210). In one embodiment, the mobileapp 150 identifies, based on the analysis, one or more relatedcharacteristics among the media items in the set to form a cluster 260of media items associated with an “event”, such as, for example, asoccer game, concert, or party (Step 215). The related characteristicscan be, for example, that two or more media items were captured within agiven time period (e.g., two digital photographs were taken within oneday or one hour of each other), two or more media items were captured atthe same or close to the same location (geo-location), two or more mediaitems have the same or similar tags associated with the media items, thesame people are present in the media item, or any other relatedcharacteristics between two or more media items in the set.

As described in more detail below, a “moment” is defined as a pluralityof media items grouped together and representing a portion of a storythat is meaningful to the user and/or to the overall context of thestory to be created.

The mobile app 150 analyzes the media items in the cluster 260 (shownwith arrow 265 in FIG. 2B) and selects, based on this analysis,templates 270 from a template database (Step 220). In one embodiment,the templates are selected by the mobile app 150 based on, for example,the layout or timing (or any other characteristic) of the media items inthe cluster 260. Each template 270 is configured to represent a momentin the story. The different templates 270 in the template database mayhave different characteristics, such as templates 270 for photographs,templates 270 for videos, templates 270 having a specific time periodallocated for different media items, templates 270 having differenttransition effects from one media item to the next media item (e.g.,fading characteristics), positioning of the media items, number of mediaitems displayed on the screen in a frame or simultaneously, etc. Thetemplate 270 can provide the level of timing in the mixed-media moduleand/or the layout of elements (e.g., based on time and/or geographicproximity).

The mobile app 150 edits selected media items in the cluster 260 to fitinto the selected templates 270 (Step 225). In one embodiment, themobile app 150 determines a desired sequence for the selected templatesand creates a Mixed-Media Module (e.g., video) 275 including thetemplates organized into the sequence (Step 230). In one embodiment, thedetermination of a desired sequence includes determining a length of theMixed-Media Module. The Mixed-Media Module 275 can be a video of thenarrative story which encapsulates the moments associated with thestory. The created Mixed-Media Module 275 includes video, digitalphotographs, music, pictures, sound, documents, text, characters, or anyother media. In one embodiment, the mobile app 150 can publish theMixed-Media Module 275 (shown with arrow 280 in FIG. 2B). Thispublishing may be automatic or may be in response to a user selection oruser action.

By way of a non-limiting simple example, suppose a user of a smartphoneuses his smartphone to take four photographs and one video of his sonduring his son's soccer game. The user also has lots of other media onhis smartphone, such as hundreds of digital photographs and videos. Thefirst two photographs of the four taken during the soccer game areduring warm-ups, the next two photographs are during the game, and thevideo occurs when the user's son scores a goal in the game. This “story”of the son's soccer game, and the media related to this story, can bemade into a Mixed-Media Module (e.g., video) by the mobile app 150,where the Mixed-Media Module includes the 4 photographs and the video.The mobile app 150 may also include a song from the user's song library(e.g., stored on the user's phone) that plays while the Mixed-MediaModule plays.

For example, the mobile app 150 can receive, from the client device 105or from another computing device (e.g., associated with the user,connected to the client device 105 via the Internet, etc.), selectionsof media items accessible to the user or user devices for use in aMixed-Media Module that tells a narrative story or event. In anotherembodiment, the mobile app 150 automatically selects the media items foruse in a specific Mixed-Media Module. This automatic selection may bebased on multiple media items being captured within a specific timeframe or having a time stamp associated with the items within thespecific time frame.

As described in more detail below, advertisement server 140 may provideone or more advertisements to the mobile device (e.g., targetedadvertisements based on the media item(s) in the Mixed-Media Module).

Referring to FIGS. 3A and 3B, in one embodiment the mobile app 150 canobtain the media items associated with the user from multiple sources.In one embodiment, the mobile app 150 analyzes a first set of mediaitems, where each media item has associated metadata (Step 305). Themobile app 150 analyzes the items and/or the metadata of each media itemin the first set, and identifies one or more related characteristicsamong the media items in the first set to form a cluster of media itemsassociated with an event (Step 310). The mobile app 150 obtains themedia items in the cluster from the client device 105 (e.g., database350) (Step 315). The obtained media items are represented in FIG. 3B asinternal media items 355.

The mobile app 150 then analyzes a second set of media items associatedwith the user, where each media item in the second set has associatedmetadata (Step 320). As described in more detail below, the second setof media items may be media items that were captured via a differentapplication on the client device 105. Specifically, in one embodimentthe mobile app 150 can monitor other applications installed on theclient device 105 and access media generated and shared within thoseapplications to create more robust, relevant, and social Mixed-MediaModules. For example, implementations of the mobile app 150 couldautomatically access a user's Facebook® or other social media photos tocreate Mixed-Media Modules.

In one embodiment, the mobile app 150 analyzes the associated metadataof each media item in the second set and identifies, based on thisanalysis, the one or more related characteristics among one or moremedia items in the second set (Step 325). The mobile app 150 obtains theone or more media items in the second set (shown in FIG. 3B as externalmedia items 365) from a remote computing device 360 associated with thesecond set (Step 330). This remote computing device 360 may be anothermobile computing device (e.g., another smartphone), a desktop or laptopcomputer storing media items, cloud storage, a plurality of computers orweb sites, etc.

The mobile app 150 adds the obtained media items in the second set tothe cluster of media items (Step 335). The mobile app 150 then analyzesthe media items in the cluster and selects, based on the analysis of themedia items in the cluster, templates from a template database (Step340). As described herein, each template is configured to represent amoment or time period or sub-topic in the event or story. Mobile app 150edits the selected media items in the cluster to fit into the selectedtemplates and determines a desired sequence for the selected templates.The mobile app 150 creates a Mixed-Media Module including the templatesorganized into the sequence (Step 345).

In one embodiment, the mobile app 150 obtains internal media items 355from a memory 350 of the client device 105. These internal media items355 can be, for example, photos or videos stored on the client device105 (e.g., in the Camera Roll of the device 105). In one embodiment, themobile app 150 obtains external media items 220 initially located on oneor more other devices (e.g., device 360) in communication with themobile client device 105.

In one embodiment, the media items in the second set are related to asocial connection of the user. Thus, the mobile app 150 can obtain theone or more (external) media items in the second set from one or moresocial media sites (e.g. Facebook®, Twitter®, Foursquare®, Instagram®,LinkedIn®, Flickr®, YouTube®, etc.). In one embodiment, the mobileclient device 105 requests permission from the other computing device orweb site to download the external media item(s) 365 from the othercomputing device 360 or site. In another embodiment, the mobile clientdevice 105 automatically downloads the external media item 365 from theother computing device 360 or site.

For example, one embodiment allows, that with just one “tap” or userinput, users create stories of the events in their life that they havecaptured through mixed mobile media. Thus, the mobile app 150 enablesstorytelling with Mixed-Media Modules that can take metadata andinformation that is recorded when a user takes photos, video, and othermultimedia on their mobile device and transmits that media into adynamic, editable story. This can be set as an automatic feature thatcreates digital stories (e.g., Mixed-Media Modules), that are ready andwaiting to view through the mobile app 150 after the user captures mediaon their mobile device. Depending on a degree of integration and/oraccess into the operating system or software or hardware of the clientdevice 105, implementations of Mixed-Media Modules can use even moredata from the mobile device 105. The mobile app 150, as a nativeapplication on the device 105, would have full access to all of the datacreated on the mobile device 105, facilitating the telling of fullerstories and more detailed Mixed-Media Module presentations. TheMixed-Media Module can include, for example, two or more of video(s),digital photograph(s), music, picture(s), sound, document(s), text,character(s), or any other media.

As indicated above, in one embodiment a Mixed-Media Module can becreated from media items associated with a social media site (e.g.,Facebook®). These media items may be uploaded to the social media siteby the user or by another individual or company (e.g., an individual orcompany that has a social connection (e.g., friend, follower,connection, etc.) with the user in the social media site, an advertiser,etc.).

In one embodiment and referring to FIG. 4, after the Mixed-Media Modulehas been created, the mobile app 150 displays a preview of theMixed-Media Module (Step 410). The mobile app 150 can also display editfunctions to the user for editing the Mixed-Media Module (e.g., one ormore media items in the video) (Step 415). The mobile app 150 receivesedit commands from the user for editing the Mixed-Media Module (Step420), and applies these edit commands to the Mixed-Media Module tocreate a final Mixed-Media Module (Step 425). The mobile app 150 thendisplays the final Mixed-Media Module to the user (Step 430) and, asstated above, may publish the final Mixed-Media Module.

The edit commands may be for one or more media items and may include,for example, a setting of a pace of the Mixed-Media Module or mediaitem(s) in the Mixed-Media Module, an edit to a media item or to aMixed-Media Module, adding a song to the Mixed-Media Module or to amedia item, receiving a title for the Mixed-Media Module or media item,receiving media and/or captions to add to the Mixed-Media Module,receiving additional moments for the Mixed-Media Module, receiving adescription for the Mixed-Media Module or media item, enabling sharingof the Mixed-Media Module or media item, receiving a tag associated withthe Mixed-Media Module or media item, cropping a media item orMixed-Media Module, fading or transitioning effects, etc.

Referring to FIGS. 5 and 6, in one embodiment the mobile app 150includes or is associated with a video engine 505 for creating theMixed-Media Module. The video engine 505 can be, for example, software,hardware, and/or firmware. The video engine 505 can include modules orlogic to create the Mixed-Media Module. In one embodiment, the videoengine 505 includes a set analyzer module 510 for analyzing a set ofmedia items associated with the user, where each media item hasassociated metadata (Step 610). The set analyzer module 510 analyzes theassociated metadata of each media item in the set. The set analyzermodule 510 transmits the analyzed media items and their associatedmetadata 515 to an identifier module 520. The identifier module 520identifies, based on the analysis of the associated metadata, one ormore related characteristics among the media items in the set to form acluster of media items associated with an event associated with the user(Step 615). The related characteristics of the media items may be, forexample, that two or more of the media items were captured within apredetermined or set time period, that two or more media items have beentagged with a common or similar tag, that two or more media items arecaptured in the same or similar location (geographic proximity), or haveimages of the same people, or other media characteristics obtainablefrom the media item and/or its metadata.

The identifier module 520 transmits the cluster 525 to the clusteranalyzer module 530, and the cluster analyzer module 530 analyzes themedia items in the cluster. The cluster analyzer module 530 transmitsthe analyzed cluster 535 to a template module 540, and the templatemodule 540 selects, based on the analysis of the media items in thecluster, templates from a template database (Step 620). Each template isconfigured to represent a moment in the event. A template, or layout,determines how the media item(s) in the cluster are positioned and/orconfigured in a layout for the video to represent the moment. Thetemplate module 540 edits or modifies the selected media items in thecluster to fit into the selected templates (Step 625). The templatemodule 540 transmits the templates 545 to a scoring module 550. Thescoring module 550 scores each selected template 545 based on themetadata associated with the selected media items in the selectedtemplate (Step 630). As described in more detail herein, the score maybe based on, e.g., the media item(s) contained in the template and thescoring module 550 may provide additional bonus value if the moment isthe highest scoring moment in the cluster. For example, as describedbelow, a portrait image may obtain a score of 1 point, a landscape imagemay obtain a score of 2 points, a video may obtain a score of 3 points,and a highest scoring moment may obtain a bonus score (e.g., 5 points).

In one embodiment, the scoring module 550 then determines, based on thescoring, a time period of display for each of the selected media itemsin the selected templates. The scoring module 550 also determines adesired sequence for the templates based on the scoring and based on thedetermined time periods of display. The scoring module 550 transmits thescored templates 555 to a Mixed-Media creation module 560. TheMixed-Media creation module 560 creates a Mixed-Media Module ofpredetermined duration including the templates organized into thedesired sequence and in accordance with the determined time periods(Step 635).

As shown in the example of FIG. 7, client device 705 may include one ormore processing units (also referred to herein as CPUs) 722, whichinterface with at least one computer bus 725. A memory 730 can bepersistent storage and interfaces with the computer bus 725. The memory730 includes RAM 732 and ROM 734. ROM 734 includes a BIOS 740. Memory730 interfaces with computer bus 725 so as to provide information storedin memory 730 to CPU 722 during execution of software programs such asan operating system 741, application programs 742, device drivers, andsoftware modules 743, 745 that comprise program code, and/orcomputer-executable process steps, incorporating functionality describedherein, e.g., one or more of process flows described herein. CPU 722first loads computer-executable process steps from storage, e.g., memory732, data storage medium/media 744, removable media drive, and/or otherstorage device. CPU 722 can then execute the stored process steps inorder to execute the loaded computer-executable process steps. Storeddata, e.g., data stored by a storage device, can be accessed by CPU 722during the execution of computer-executable process steps.

Persistent storage medium/media 744 is a computer readable storagemedium(s) that can be used to store software and data, e.g., anoperating system and one or more application programs. Persistentstorage medium/media 744 can also be used to store device drivers, suchas one or more of a digital camera driver, monitor driver, printerdriver, scanner driver, or other device drivers, web pages, contentfiles, playlists and other files. Persistent storage medium/media 706can further include program modules and data files used to implement oneor more embodiments of the present disclosure.

For the purposes of this disclosure a computer readable medium storescomputer data, which data can include computer program code that isexecutable by a computer, in machine readable form. By way of example,and not limitation, a computer readable medium may comprise computerreadable storage media, for tangible or fixed storage of data, orcommunication media for transient interpretation of code-containingsignals. Computer readable storage media, as used herein, refers tophysical or tangible storage (as opposed to signals) and includeswithout limitation volatile and non-volatile, removable andnon-removable media implemented in any method or technology for thetangible storage of information such as computer-readable instructions,data structures, program modules or other data. Computer readablestorage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM,flash memory or other solid state memory technology, CD-ROM, DVD, orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other physical ormaterial medium which can be used to tangibly store the desiredinformation or data or instructions and which can be accessed by acomputer or processor.

Client device 705 can also include one or more of a power supply 726,network interface 750, audio interface 752, a display 754 (e.g., amonitor or screen), keypad 756, illuminator 758, I/O interface 760, ahaptic interface 762, a GPS 764, a microphone 767, a video camera,TV/radio tuner, audio/video capture card, sound card, analog audio inputwith A/D converter, modem, digital media input (HDMI, optical link),digital I/O ports (RS232, USB, FireWire, Thunderbolt), expansion slots(PCMCIA, ExpressCard, PCI, PCIe).

For the purposes of this disclosure a module is a software, hardware, orfirmware (or combinations thereof) system, process or functionality, orcomponent thereof, that performs or facilitates the processes, features,and/or functions described herein (with or without human interaction oraugmentation). A module can include sub-modules. Software components ofa module may be stored on a computer readable medium. Modules may beintegral to one or more servers, or be loaded and executed by one ormore servers. One or more modules may be grouped into an engine or anapplication.

FIG. 8 is a block diagram illustrating an internal architecture of anexample of a computer, such as server computer 130 and/or client device105, in accordance with one or more embodiments of the presentdisclosure. A computer as referred to herein refers to any device with aprocessor capable of executing logic or coded instructions, and could bea server, personal computer, set top box, tablet, smart phone, padcomputer or media device, to name a few such devices. As shown in theexample of FIG. 8, internal architecture 800 includes one or moreprocessing units (also referred to herein as CPUs) 812, which interfacewith at least one computer bus 802. Also interfacing with computer bus802 are persistent storage medium/media 806, network interface 814,memory 804, e.g., random access memory (RAM), run-time transient memory,read only memory (ROM), etc., media disk drive interface 808 as aninterface for a drive that can read and/or write to media includingremovable media such as floppy, CD-ROM, DVD, etc. media, displayinterface 810 as interface for a monitor or other display device,keyboard interface 816 as interface for a keyboard, pointing deviceinterface 818 as an interface for a mouse or other pointing device,CD/DVD drive interface 820, and miscellaneous other interfaces (e.g.,interface 822) not shown individually, such as parallel and serial portinterfaces, a universal serial bus (USB) interface, and the like.

Memory 804 interfaces with computer bus 802 so as to provide informationstored in memory 804 to CPU 812 during execution of software programssuch as an operating system, application programs, device drivers, andsoftware modules that comprise program code, and/or computer-executableprocess steps, incorporating functionality described herein, e.g., oneor more of process flows described herein. CPU 812 first loadscomputer-executable process steps from storage, e.g., memory 804,storage medium/media 806, removable media drive, and/or other storagedevice. CPU 812 can then execute the stored process steps in order toexecute the loaded computer-executable process steps. Stored data, e.g.,data stored by a storage device, can be accessed by CPU 812 during theexecution of computer-executable process steps.

As described above, persistent storage medium/media 806 is a computerreadable storage medium(s) that can be used to store software and data,e.g., an operating system and one or more application programs.Persistent storage medium/media 806 can also be used to store devicedrivers, such as one or more of a digital camera driver, monitor driver,printer driver, scanner driver, or other device drivers, web pages,content files, playlists and other files. Persistent storagemedium/media 806 can further include program modules and data files usedto implement one or more embodiments of the present disclosure.

Internal architecture 800 of the computer can include (as stated above),a microphone, video camera, TV/radio tuner, audio/video capture card,sound card, analog audio input with A/D converter, modem, digital mediainput (HDMI, optical link), digital I/O ports (RS232, USB, FireWire,Thunderbolt), and/or expansion slots (PCMCIA, ExpressCard, PCI, PCIe).

In more detail, the mobile app 150, as a native application to themobile device 105, has more extensive use of the mobile device'sresources including memory, processor, permissions to other nativeapplications, and further access to enable implementations of the mobileapp 150 to add more sophisticated features like: enhanced notifications,pre-rendering and uploading of media “off screen”, etc. All of theseoperations would help create better Mixed-Media Modules more efficientlyby accessing the users' real-life timeline.

In one embodiment, the mobile app 150 enables iOS and Android users, forexample, or any other kind of mobile operating system, to tell storiesin one tap. Implementations herein, accordingly, make storytelling on amobile device 105 quick and allow users to share events in beautifulpresentations (Mixed-Media Modules 275) (e.g., as they are happening).

As described above, the mobile app 150 can, in one embodiment,automatically turn collections of mixed media items created on a mobiledevice 105 into presentations 275 using metadata and other propertiesstored in photos and videos from the user's mobile device library. Forexample, one embodiment enables, with just one tap of their finger,users to create stories of the events in their life that they havecaptured through mobile media. By simplifying storytelling and socialcollaboration on mobile devices, the mobile app 150 can, via the createdMixed-Media Module, deepen a user's ties to the captured events and thepeople in them. The mobile app 150 with Mixed-Media Modules make iteasier to digitally document and broadcast experiences, not just staticmoments in time.

In one embodiment, the mobile app 150 can create a Mixed-Media Module275 automatically, so that they are ready and waiting to view throughthe mobile app 150 after the user captures media items on their mobiledevice 105. In one embodiment, the mobile app 150 is configured forcustomization. This means that authors of Mixed-Media Modules 275 canmake different projects distinctive on their mobile device 105. In someimplementations, for example, authors/users can use the mobile app 150to form their captured media items into an interactive, professionallooking video presentation 275 that tells the story of an event ormoment without the time, expenses or limitations of current videoediting technology.

In further implementations, the processing of the animation for theMixed-Media Module 275 may also be off-loaded to the device 105 playingback the Mixed-Media Module 275, which means bandwidth is preserved.According to some exemplary embodiments, the animation is defined in atext-based format as opposed to a binary data format, making it moreefficient and entirely cross-platform (e.g., IOS, Android, Blackberry,etc.).

Within each event, in one embodiment the mobile app 150 can coordinatethe display of pictures and videos taken within relevant timings of eachother for storytelling and aesthetic impact. This is the “moment”—thatis, the level of timing in the presentation 275, and the layout ofelements, based on, e.g., time and geographic proximity.

In one embodiment, the mobile app 150 can enable users to tag venues,and animate the media recorded at those venues alongside data on thevenue's address, ratings, etc. The mobile app 150 may also utilizemoment-level metadata to assemble animations (for instance, a user canshow a restaurant's venue animation prior to all the photos taken at therestaurant). With full access to other applications native to the device105, implementations of the mobile app 150 can use data from thoseapplications to create context around events and tell a more completestory.

Advertising can be targeted more effectively, as a Mixed-Media Moduleauthor may have control over the types of advertising viewed by theiraudience, or the ad insertion could be automatic. The mobile app 150,when native to the mobile device 105, can access other applications onthat mobile device 105 and can use that shared metadata to identify andapply valuable information to targeting relevant advertising. In thisway, the systems and methods herein may also involve a Mixed-MediaModules ad suggestion engine that enhances the advertising experiencefor both the Mixed-Media Modules author and the audience. Inembodiments, metadata can be analyzed to extract or form ad keywordsthat can be fed to an ad server or ad platform to result incontext-relevant ads being included before, during, or after theMixed-Media Module 275.

In still further implementations, ads on Mixed-Media Modules 275 may beconfigured like ads in a magazine—selected by the author or publisher tofit the audience and the adjacent content. As such, the added contextand relevance of ads may result in a higher CPM for advertisers who willpay higher rates for better results and more targeted viewers. Also,features herein allow advertising and interactive advertising to beintegrated seamlessly without interrupting the viewing experience and sopreserving the quality and user experience of the Mixed-Media Modulespresentations 275.

The mobile app 150 can, in one embodiment, determine how the user of themobile device 105 uses their device 105. As described above, this meansthat the mobile app 150 can monitor other applications installed on thedevice 105 and access media generated and shared within thoseapplications to create more robust, relevant, and social Mixed-MediaModules 275.

For example, implementations of the mobile app 150 can automaticallyaccess a user's Facebook® photos to create Mixed-Media Modules 275. Thiscould include the photos the user has uploaded to Facebook®, or othersocial media site, and photos the user has been tagged in by theirFacebook® friends. Access to the historic media roll of other socialnetworks and the associated metadata can be used to improve therelevance and robustness of the presentations 275 by enabling theauto-reference of locations and people who make moments meaningful.

In some implementations, the mobile app 150 can render raw video filesusing the mobile device 105, but the user must be “on screen” in theapplication 150. In other implementations, the mobile app 150 performsthis operation even while the user is “off screen” in anotherapplication. This can improve the efficiency of creating Mixed-MediaModules 275, saving the users time and allowing them to create morestories using the system. For example, implementations of theMixed-Media Modules mobile app 150 can render a video 275 on the mobilehandset allowing the user to then upload that video 275 to YouTube®,Facebook®, or any other website to syndicate to a larger audience.

By adding background operations post-rendering, in one embodiment themobile app 150 can improve viral sharing based on user preferences.Instead of manually uploading the newly rendered Mixed Media Module 275to social media and video hosting website(s), implementations of themobile app 150 can auto-upload rendered Mixed Media Modules 275 towebsites based on permissions set by the user. Specifically, forexample, the mobile app 150 can auto-upload that Mixed Media Module 275to YouTube®, Facebook®, Twitter®, or any other website to syndicate to alarger audience by allowing the user to set that permission and linktheir YouTube®, Facebook®, Twitter® or other social media account to theMixed-Media Module and/or mobile app 150.

Exemplary implementations may be configured to allow users to open theapp 150, and be greeted with a list of new events identified, and a usercan be prompted to publish them as independent, interactivepresentations 275. This is an embodiment of a notification to publishautomatically clustered moments. The mobile app 150 may also beconfigured to allow for the display of an automatically created draftwith one-tap customization. Again, if a user wished to customize, theycan do so via a simple tagging interface. Systems and methods herein mayalso be configured to allow for tagging of metadata from authorizedapplications (friends, locations) to generate animations. Title,location, venue, friends and other identifying animations are generatedfrom metadata attached to the photos—either from the camera roll, orthird party social network sources like Facebook® and Foursquare®.

The mobile app 150 may also allow the user to apply multiple filters tomultiple videos and images at a single time. According to systems andmethods herein, users don't simply select filters—they set styles, whichare combinations of image filter, video filter, font and caption effectsthat apply to the media in a given Mixed-Media Module 275. There areoptional style plug-ins users can choose to emulate other famous styles.This helps the user easily create context linking of their media inorder to tell a story.

The mobile app 150 can also be used to collaborate with friends after anevent to share captured media and create shared stories in the form ofassociated Mixed-Media Module implementations configured for suchutilization. In one embodiment, one or more of the following steps areperformed:

-   -   1) After the event, a user can define the event by entering a        start date/time and an end date/time, and can then blast a        message to friends asking if they'd like to share their media        from this event. All the friends have to do is agree (e.g., tap        YES) to include their media in the user's presentation/module        (Mixed-Media Module) 275.    -   2) The same process may also be configured to execute        automatically (e.g., the mobile app 150 ascertains that there        was an event that took place on day X from Y am to Z pm, and        prompts the user to ask friends for media).    -   3) A user's friends grant the processing component(s) access to        their camera roll, and as the user creates Mixed-Media Modules        275, the mobile app 150 automatically searches their camera        rolls for relevant media items and adds them to the user's        Mixed-Media Module 275.

Implementations herein can turn collections of mixed media into astoryline or narrative presentation using metadata and other propertiesstored in photos and videos from the user's media library. The followingare sets of optional rules or strategies that can help arrange andformat the presentations. Combinations of the rules and strategies canbe employed in different presentations. Not all are necessary to be usedeach time and any combination can be used. In one embodiment, one ormore of the following steps are performed:

(1) Identification of the important events in the user's life, such asvia processing/organizing event-level “clusters” of media based on timeand geographic proximity.

(2) Generation of moment-level timing and/or layout of the presentationbased, e.g., on time and geographic proximity, and which may also becoordinated for maximum storytelling and aesthetic impact.

(3) Assigning animations and/or pacing of presentation, e.g., usingmoment-level metadata from application(s) of authorized device(s) formaximum storytelling and aesthetic impact.

Implementations herein may include or involve a clustering algorithmthat identifies the events in the user's life based around the amount ofmedia captured on a mobile device in a certain time and place, thenclusters this media and presents these events for playback and editingas a self-contained Mixed-Media Module 275.

The implementation/application accesses and analyzes photos from theuser's camera roll or media library or authorized applications enabledby, for example, full stack access.

Processing herein can then utilize a time range as the bounds forselection, automatically extracting events from the set of mixed mediaitems. Some implementations may be configured such that there are threeincluded time ranges, and may also be configured such that users maydefine custom time spans as shown, for example, in FIG. 9 with a StoryRank Algorithm 900. The Story Rank Algorithm 900 can include differentspeeds 905 for different media types and different scenarios. FIG. 9also shows a time allocation worksheet 910 which illustrates differentspeeds for a portrait media 915, a landscape media 920, and videos 925.

Examples may include:

-   -   Day: identified as 5:01 am to 5:00 am the following day    -   Weekend: identified as 6:00 pm on a Friday to Monday at 5:00 am    -   Special event: identified as any 2-hour period where more than 9        pieces of media are captured

A user can manually create a Mixed-Media Module 275 by accessing theirmobile device's photo library or media from authorized applications andselecting a set of mixed media items to include. For example, somesystems may be configured to allow a user to select up to 30 pieces ofmixed media to include or some greater or smaller number. Suchimplementations/processes can create a story from that set of mediaitems and apply the same rules to the generation process.

Given the media from an event, the mobile app 150 can then group theminto moments according to rules around the frequency in which it wastaken. An example goal of this process is to use the timeline the userimplicitly creates while capturing media to inform the creation of alinear story composed of audio and visual components.

In one embodiment, the mobile app 150 may be configured to find thetime-delta between capture of each media item. It can then obtain theaverage time-delta between each media item. Using the averagetime-delta, it can split media items into moments. It then iteratesacross each piece of media and, when a time-delta of capture time isdetected that is longer than the average time-delta, the mobile app 150can group the preceding media into a moment.

In some implementations and as introduced above, for ranking purposes,each moment can be given a score based on the media item(s) contained inthe moment and an additional bonus value if the moment is the highestscoring moment. For example:

-   -   Portrait image: 1 point    -   Landscape image: 2 points    -   Video: 3 points    -   Highest scoring moment bonus: 5 points

The mobile app 150 may be configured to identify display speed, andbased on the moment's scores, the Mixed-Media Module 275 can be applieda display speed. Mixed-Media Module configurations herein may have manyspeeds, such as, for example, three display speeds: “Fast”, “Medium” or“Slow”, each of which has default timing values for each type of media.Note these scores can be further modified via a granular “slider”control bar exposed to the user, allowing for custom speed application.For example,

-   -   I. If there are two or more moments with a score of 10 or above,        the Mixed-Media Module may be set/configured to “Fast”    -   II. If there is one moment with a score of 10 or above, the        Mixed-Media Module may be set/configured to “Medium”    -   III. All others scores, Mixed-Media Module is “Slow”        -   Configurations also include other speeds, which may also be            customized, customizable and/or changed.

“Fast” “Medium” “Slow” Portrait 0.75 seconds 1.25 seconds 1.75 secondsImage Landscape 1 second 1.75 seconds 2.25 seconds Image Video 4 seconds  5 seconds   6 secondsFurther configurations include, if the Mixed-Media Module play time isgreater than a predetermined time (e.g., 30 seconds), group media withinmoments. If a Mixed-Media Module play time exceeds its predeterminedtime limit (e.g., 30-second limit), media within moments can becollapsed upon each other and displayed within the same timespan via avariety of layouts (often referred to as “grids”). This allows for moremedia to be displayed in less time. It begins by collapsing the lowestscoring moment into multimedia layouts. If the Mixed-Media Module playtime is still less than its limit (e.g., 30-seconds), continue theprevious step until the Mixed-Media Module play time is less than orequal to its limit (e.g., 30 seconds). Videos are also being processedto increase display speed (via fast forward) or decrease display speed(via slow-motion) to match the relative fast, medium, or slow pacing ofthe Mixed-Media Module playback.

Implementations may be configured such that a user can choose layouts(e.g., templates) based on the collections of media within moments.There may be numerous layout styles that are targeted towardscollections of 1-4 pieces of media and containing different animationstyles. These layout styles are shown, e.g., in layout styles 1005,1010, and 1020 of FIGS. 10A, 10B, and 10C, respectively. Media groupedin layouts can be displayed all at once in a “total reveal” or insequence in a “cascading reveal” as shown, for example, in FIGS. 10A,10B, and 10C.

In one embodiment, the mobile app 150 applies dynamic pacing forstorytelling. For example, the first media, if it's an image, isdisplayed twice as long. The last media, if it's an image, can bedisplayed, e.g., twice as long. The highest scoring moment gets applieda unique treatment which can include, e.g.:

-   -   Adjusting display times of media relative to the Mixed-Media        Module play back: faster or slower    -   Unique transition effects in between or during media display    -   Applying a unique layout of the media that is not used elsewhere        during playback        In one embodiment, escalation of effects preceding the highest        scoring moment means that moments immediately preceding the        highest scoring moment are applied unique transitions to signal        the upcoming unique display properties of the highest scoring        moment.

It is contemplated that a song (or songs) will be incorporated into thestory of the Mixed-Media Module as a soundtrack. Songs can be foundamong a user's available media or downloaded from network accessiblesources. In embodiments, the song title or genre is selected to matchthe story context. If beats per minute (BPM) for the song is known, themobile app 150 can augment the display times for media to compensate forthe beat of the song, matching to whole beats, half beats, or quarterbeats, thus matching the transitions. In one embodiment, thegeographical & time data is used to set titles for implementations ofthe Mixed-Media Modules 275.

In exemplary implementations, the mobile app 150 can actively monitor auser's photo library from their mobile device, and, upon detection of aset of media that corresponds to an existing Mixed-Media Module 275,through using the rules above, that Mixed-Media Module presentation 275can be generated. The user can be notified of the creation of aMixed-Media Module 275 using one or more messages.

In one embodiment, shown in play mixed-media module screenshot 1100 ofFIG. 11, tapping a play button 1105 can play Mixed-Media Module 1110. Inone embodiment, tapping a magnifying glass icon 1120 can put theMixed-Media Module 1105 in preview mode. The user can view, for example,comments, users who liked the presentation, and additional otherMixed-Media Modules 1105 associated with the tagged location.

FIG. 12 shows an example preview screenshot 1200 where, in oneembodiment, Mixed-Media Modules 275 are labeled as previews 1205, 1210when they are unpublished. In one embodiment, tapping the magnifyingglass switches the stack of media to preview mode.

FIG. 13 shows an example user editing screenshot 1300 where the user canedit a Mixed-Media Module 1305. The user can, for example, change theMixed-Media Module's filter, edit moments, add a song and set the paceof the slideshow. The pace may be set inline via a slider. In oneembodiment, the user can set the title of a Mixed-Media Module 275. Theuser can edit specific moments by adding media and captions. In oneembodiment, the user can add additional moments, can provide adescription, and/or can share the Mixed-Media Module before publishingit. In one embodiment, the location can be changed and/or friends can betagged. The Mixed Media Module 275 can be shared or posted to a socialmedia site. In one embodiment, the user can trim or crop videos orphotos (e.g., via the standard iOS interface). In one embodiment, theuser can add a song from their library or by searching iTunes®. The usercan trim the song (e.g., via a draggable slider). The user can tagusers, and/or can select a location of the media item or Mixed MediModule 275.

In one embodiment, the mobile app 150 shows a map indicating where theapp 150 drew the media from to create the Mixed-Media Module 275. Forexample, the geographic locations can be shown on the map withindicators identifying the number of media items that relate to aspecific location.

In one embodiment, the mobile app 150 enables social, onlineinteraction. For example, the user may have a profile, can followothers, and can be followed by others. Further, in one embodiment, theuser's Mixed Media Modules can be “liked” and other users (or the user)can provide comments (e.g., on, about, or related to the Module). Themobile app 150 can allow for collaboration with social media contactsand friends to share and publish captured media at events to createshared videos that tell stories in the form of the Mixed-Media Modules275. Users can collaborate, create and share Mixed-Media Modules 275both before an event and after an event all on their mobile device 105.

The ability to share captured multimedia and create Mixed-Media Modules275 with friends can be done during an event and after the event,gradually building up to the peak experience that it can be shared withfriends. For example, one or more of the following steps can beperformed (See, e.g., FIGS. 14-20):

-   -   1. The owner or administrator of a presentation associated with        a desired activity or event can send an invitation to join in        creating a Mixed-Media Module 275 to their friends. This can be        done a number of ways. Some of these include sending an        invitation via social media websites, such as Facebook®,        Twitter®, email, or SMS to have another person download the        mobile app 150, if they do not already have it on their mobile        device. This enables the individual to, during the event,        capture media on their mobile device to be shared and integrated        automatically or manually into Mixed-Media Modules 275. This can        be done on a Create home page (e.g., screenshot 1400 of FIG. 14)        on the mobile app 150 where there are options on the screen to        create a Mixed-Media Module “From my Camera Roll”, “With        Friends”, “With All”, or from other sources, for example.    -   2. With the “From my Camera Roll” option, the mobile app 150        allows the creator to begin adding media from their camera roll        on their mobile device 105 to begin the creation of the        Mixed-Media Module 275. The “With Friends” option invites the        user's friends (from the app), or can also integrate with other        friend lists from various sources, such as social media        applications including but not limited to Facebook®, Twitter®,        and Instagram® (See, e.g., screenshot 1500 of FIG. 15).    -   3. The “With All” option allows creation of a Mixed-Media Module        275 as a public event for open collaboration with the community.        This can enable a mobile app user to connect with others using        the app 150 at an event and share their media collectively to        create a Mixed-Media Module 275.    -   4. The user attends the event and announces their arrival        through the mobile app 150 or through invitations, and/or        sending off notifications for others to join. The user is also        able to see other users that are in proximity to them through        geo-location, and they are able to invite others to join through        this channel as well. If the owner of the Mixed-Media Module 275        chooses to create with friends, in the next screen, they select        friends (simply searching their friend list on the app/phone        book/Facebook®/other sources (for example), and send invites        such as the invite screenshot 1500 shown in FIG. 15. In one        embodiment, the user waits for friends to contribute.    -   5. The friend or mobile app user can receive an invitation via a        notification or alert on their device (See, e.g., screenshot        1600 of FIG. 16).    -   6. If the owner or administrator of a Mixed-Media Module chooses        to create a public event for open collaboration they would        choose the “Create With All” option (See, e.g., screenshot 1700        of FIG. 17). On this screen, they can, e.g., pick a hash tag to        label the event and, in this case, can add users to the public        event. Once a user's friends are notified who is at the event        and is in the group granted access to contribute to the        Mixed-Media Module 275, they can opt in and begin sharing their        captured media to be made into implementations of Mixed-Media        Modules 275 (See, e.g., screenshot 1800 of FIG. 18).    -   7. At this point the user that was invited and has opted into        the particular Mixed-Media Module 275 can begin sharing uploads        of their media, and this subsequently allows the owner of the        event (who originally invited this user) to see their added        media in the screen.    -   8. Again, users have a choice of which multimedia they share        with the group creating the Mixed-Media Module 275 and can        choose to create independent Mixed-Media Modules 275 on their        own from the shared multimedia. This can all be done in        real-time and Mixed-Media Modules 275 can be created during an        event and can added to until there is a finished Mixed-Media        Module 275 at the end of an event (See, e.g., screenshot 1900 of        FIG. 19).    -   9. During and after the event, the Mixed-Media Module 275 can be        previewed and edited. When the owner or administrator of the        Mixed-Media Module 275 is ready, they can “end” the event and        all of the submitted media can then be clustered and output into        the Mixed-Media Module format.    -   10. Users can then edit and publish the Mixed-Media Module of        the event, with the contributors credited.    -   11. There are also notifications that alert users who have        contributed to a Mixed-Media Module to view it. In one        embodiment, this is a standard system notification the user can        receive outside of the app on their mobile device (See, e.g.,        screenshot 2000 of FIG. 20).

In one embodiment, a user opens the mobile app 150 and is presented witha default list of media presentations (Qwikis) that are currentlypopular or from the profiles of other users they follow. The userscrolls through the Qwikis which are presented (e.g., as individualthumbnails) to read the titles, preview, and, as they pause on athumbnail with their finger while scrolling down, the associated dataanimates out and they can watch the Qwiki, comment on it, see who elsehas commented, share it, and see the soundtrack. It's a highly efficientway to share a feed of videos and preview videos without having to watchone. By not opening each multi-media presentation, this preserves databandwidth and becomes a more efficient way of viewing.

Those skilled in the art will recognize that the methods and systems ofthe present disclosure may be implemented in many manners and as suchare not to be limited by the foregoing exemplary embodiments andexamples. In other words, functional elements being performed by singleor multiple components, in various combinations of hardware and softwareor firmware, and individual functions, may be distributed among softwareapplications at either the user computing device or server or both. Inthis regard, any number of the features of the different embodimentsdescribed herein may be combined into single or multiple embodiments,and alternate embodiments having fewer than, or more than, all of thefeatures described herein are possible. Functionality may also be, inwhole or in part, distributed among multiple components, in manners nowknown or to become known. Thus, myriad software/hardware/firmwarecombinations are possible in achieving the functions, features,interfaces and preferences described herein. Moreover, the scope of thepresent disclosure covers conventionally known manners for carrying outthe described features and functions and interfaces, as well as thosevariations and modifications that may be made to the hardware orsoftware or firmware components described herein as would be understoodby those skilled in the art now and hereafter.

While the system and method have been described in terms of one or moreembodiments, it is to be understood that the disclosure need not belimited to the disclosed embodiments. It is intended to cover variousmodifications and similar arrangements included within the spirit andscope of the claims, the scope of which should be accorded the broadestinterpretation so as to encompass all such modifications and similarstructures. The present disclosure includes any and all embodiments ofthe following claims.

What is claimed is:
 1. A method comprising: analyzing, by a processor ofa computing device operated by a user, a set of media items associatedwith the user, each media item having associated metadata; identifying,by the processor, based on analysis of the associated metadata, one ormore related characteristics among the media items in the set to form acluster of media items associated with an event associated with theuser; selecting, by the processor and based on analysis of the mediaitems in the cluster, a plurality of templates from a template database,each template of the plurality of templates configured to represent amoment in the event; editing, by the processor, selected media items inthe cluster to fit into the selected templates; creating, by theprocessor, a mixed-media module comprising the plurality of templatesorganized into a desired sequence for the selected templates;displaying, by the processor to the user, a preview of the mixed-mediamodule; receiving, by the processor, edit commands from the user forediting the mixed-media module; applying, by the processor, the editcommands to the mixed-media module to create a final mixed-media module;and displaying, by the processor, the final mixed-media module.
 2. Themethod of claim 1, further comprising analyzing, by the processor, theassociated metadata of each media item in the set.
 3. The method ofclaim 1, further comprising obtaining, by the processor, the media itemsin the set.
 4. The method of claim 1, further comprising analyzing, bythe processor, the media items in the cluster.
 5. The method of claim 1,further comprising determining, by the processor, the desired sequencefor the selected templates.
 6. The method of claim 1, further comprisingdisplaying, by the processor to the user, edit functions for editing themixed-media module.
 7. The method of claim 1, wherein the receiving ofthe edit commands further comprises receiving a pace of the mixed-mediamodule.
 8. The method of claim 1, wherein the receiving of the editcommands further comprises receiving a song or music.
 9. The method ofclaim 1, wherein the receiving of the edit commands further comprisesreceiving one or more of a caption, a comment, and a title of themixed-media module.
 10. A computing device comprising: a processor; astorage medium for tangibly storing thereon program logic for executionby the processor, the program logic comprising: analyzing logic executedby the processor for analyzing a set of media items associated with auser, each media item having associated metadata; identifying logicexecuted by the processor for identifying, based on analysis of theassociated metadata, one or more related characteristics among the mediaitems in the set to form a cluster of media items associated with anevent associated with the user; selecting logic executed by theprocessor for selecting, based on analysis of the media items in thecluster, a plurality of templates from a template database, eachtemplate of the plurality of templates configured to represent a momentin the event; editing logic executed by the processor for editingselected media items in the cluster to fit into the selected templates;creating logic executed by the processor for creating a mixed-mediamodule comprising the plurality of templates organized into a desiredsequence for the selected templates; displaying logic executed by theprocessor for displaying to the user a preview of the mixed-mediamodule; receiving logic executed by the processor for receiving editcommands from the user for editing the mixed-media module; applyinglogic executed by the processor for applying the edit commands to themixed-media module to create a final mixed-media module; and displayinglogic executed by the processor for displaying the final mixed-mediamodule.
 11. The computing device of claim 10, further comprisingmetadata analyzing logic executed by the processor for analyzing theassociated metadata of each media item in the set.
 12. The computingdevice of claim 10, further comprising obtaining logic executed by theprocessor for obtaining the media items in the set.
 13. The computingdevice of claim 10, further comprising cluster analyzing logic executedby the processor for analyzing the media items in the cluster.
 14. Thecomputing device of claim 10, further comprising determining logicexecuted by the processor for determining the desired sequence for theselected templates.
 15. The computing device of claim 10, furthercomprising edit function displaying logic executed by the processor fordisplaying, to the user, edit functions for editing the mixed-mediamodule.
 16. The computing device of claim 10, wherein the receivinglogic for receiving the edit commands further comprises receiving logicfor receiving one or more of a pace of the mixed-media module, a song,music, a caption, a comment, and a title of the mixed-media module. 17.A server computer comprising: a processor; a storage medium for tangiblystoring thereon program logic for execution by the processor, theprogram logic comprising: application communication logic executed bythe processor for communicating a software application to a computingdevice operated by a user, the software application configured to:analyze a set of media items associated with the user, each media itemhaving associated metadata; identify, based on analysis of theassociated metadata, one or more related characteristics among the mediaitems in the set to form a cluster of media items associated with anevent associated with the user; select, based on analysis of the mediaitems in the cluster, a plurality of templates from a template database,each template of the plurality of templates configured to represent amoment in the event; edit selected media items in the cluster to fitinto the selected templates; create a mixed-media module comprising theplurality of templates organized into a desired sequence for theselected templates; display, to the user, a preview of the mixed-mediamodule; receive edit commands from the user for editing the mixed-mediamodule; apply the edit commands to the mixed-media module to create afinal mixed-media module; and display the final mixed-media module. 18.A non-transitory computer readable storage medium tangibly storingcomputer program instructions capable of being executed by a computerprocessor, the computer program instructions defining the steps of:communicating, by the computer processor, a software application to acomputing device operated by a user, the software application configuredto: analyze a set of media items associated with the user, each mediaitem having associated metadata; identify, based on analysis of theassociated metadata, one or more related characteristics among the mediaitems in the set to form a cluster of media items associated with anevent associated with the user, select, based on analysis of the mediaitems in the cluster, a plurality of templates from a template database,each template of the plurality of templates configured to represent amoment in the event; edit selected media items in the cluster to fitinto the selected templates; create a mixed-media module comprising theplurality of templates organized into a desired sequence for theselected templates; display, to the user, a preview of the mixed-mediamodule; receive edit commands from the user for editing the mixed-mediamodule; apply the edit commands to the mixed-media module to create afinal mixed-media module; and display the final mixed-media module. 19.A method comprising: communicating, by a processor, a softwareapplication to a computing device operated by a user, the softwareapplication configured to: analyze a set of media items associated withthe user, each media item having associated metadata; identify, based onanalysis of the associated metadata, one or more related characteristicsamong the media items in the set to form a cluster of media itemsassociated with an event associated with the user, select, based onanalysis of the media items in the cluster, a plurality of templatesfrom a template database, each template of the plurality of templatesconfigured to represent a moment in the event; edit selected media itemsin the cluster to fit into the selected templates; create a mixed-mediamodule comprising the plurality of templates organized into a desiredsequence for the selected templates; display, to the user, a preview ofthe mixed-media module; receive edit commands from the user for editingthe mixed-media module; apply the edit commands to the mixed-mediamodule to create a final mixed-media module; and display the finalmixed-media module.
 20. The method of claim 19, wherein the softwareapplication is further configured to: analyze the associated metadata ofeach media item in the set; obtain the media items in the set; analyzethe media items in the cluster, determine the desired sequence for theselected templates; and display, to the user, edit functions for editingthe mixed-media module.